Growing evidence suggests that large language models do not use their depth uniformly, yet we still lack a fine-grained understanding of their layer-wise prediction dynamics. In this paper, we trace the intermediate representations of several open-weight models during inference and reveal a structured and nuanced use of depth. Specifically, we propose a "Guess-then-Refine" framework that explains how LLMs internally structure their computations to make predictions. We first show that the top-ranked predictions in early LLM layers are composed primarily of high-frequency tokens, which act as statistical guesses proposed by the model early on due to the lack of appropriate contextual information. As contextual information develops deeper into the model, these initial guesses get refined into contextually appropriate tokens. Even high-frequency token predictions from early layers get refined > 70% of the time, indicating that correct token prediction is not "one-and-done". We then go beyond frequency-based prediction to examine the dynamic usage of layer depth across three case studies. Together, our results provide a detailed view of depth usage in LLMs, shedding light on the layer-by-layer computations that underlie successful predictions and providing insights for future works to improve computational efficiency in transformer-based models. Despite the remarkable performance of large language models (LLMs), their internal computations remain poorly understood. One critical question is: how do LLMs internally structure their computations during inference and use their depth layer-by-layer to arrive at predictions? Are specific token predictions always computed at the last layer or does the model settle on predictable tokens early on and simply propagate these predictions? These questions have implications both for interpreting the internal computations of these models and for building more efficient LLM that can use their compute dynamically.

An interactive map has revealed a disturbing pattern in drone sightings across the US. An unexplained drone invasion has targeted America's military bases worldwide since October, beginning with a swarm over Langley Air Force Base in Virginia. The pattern became evident when similar activity was reported over New Jersey's Picatinny Arsenal on November 18. Less than one week later, US bases in England and Germany began grappling with incursions by'small unmanned aerial systems.' Back in America sightings were gaining traction. 'Multiple' instances of drones appeared over New Jersey's Navy weapons station, and Ohio's Wright-Patterson Air Force Base closed its airspace due to similar activity on December 13.

The Federal Aviation Administration issued temporary flight restrictions prohibiting drone flights over parts of New Jersey following an influx of sightings in recent weeks. The notice, which expires Jan. 17, 2025, said drone operations in support of national defense, homeland security, law enforcement, firefighting, search and rescue or disaster response missions are not included in the restrictions. Commercial drone operations are allowed with a valid statement of work, but there must be an approved special governmental interest airspace waiver and all applicable FAA regulations must be followed. House Speaker Mike Johnson, R-La., said the White House, and more broadly the U.S. government, does not seem concerned about the increased sightings in New Jersey and other northeastern states. "Look, I'm the speaker of the House. I have the exact same frustrations that you do and all of us do. We don't have the answers. The administration is not providing them," Johnson said in a Fox News appearance.

Suspicious drone sightings in states across the Northeast in recent weeks may be coming from inside the United States, according to a military expert. Civilians and lawmakers have reported drone sightings in New Jersey, Pennsylvania, New York, Connecticut, Ohio and other states, with local and federal law enforcement offering little information about the aerial activity, most of which has been spotted at night. Some of the drones are as large as 6 feet in diameter, according to New Jersey state Rep. Dawn Fantasia, who was briefed on the matter last week. "The concern is definitely valid. One thing I do believe, I believe the government knows the source of these drones, and I believe the source of these drones is from inside the U.S., especially the larger drones," Col. William Dunn, president of Strategic Resilience Group, a government consulting group, told Fox News Digital.

White House National Security Communications Advisor John Kirby responds to more questions over the aerial systems on'The Story.' White House National Security Communications Advisor John Kirby maintained that the government still lacks definitive answers regarding the nature of reported drone sightings as public frustration intensifies. "Many of the corroborated sightings have turned out to be piloted aircraft. I didn't say all of them, and what I said was those are the ones we were able to corroborate," Kirby said on "The Story." "There certainly is ones that we have not been able to, and we don't know the answer to it, and I strongly recommend that for folks that are seeing these things and documenting them to share that as they can with the Department of Homeland Security and the FBI." In a Wednesday letter to Biden, New Jersey Gov. Phil Murphy asked the president for more federal resources to address drone sightings, noting that the federal law limits the ability of state and local law enforcement to counter drones.

A spectre is haunting the United States--the spectre of drone warfare. Since the middle of November, unidentified unmanned aerial vehicles have lit up the skies above New Jersey, startling residents and baffling military and government officials. The US Army's Picatinny Arsenal research and manufacturing facility in the state's Morris County reported 11 confirmed instances of mysterious drones illegally entering its airspace since the middle of the month, while a dozen drones were spotted hovering over US Naval Weapons Station Earle in Monmouth County in early December. Similar sightings were reported in at least six other counties throughout the state; according to the Coast Guard, a group of drones even followed one of the service's vessels "in close pursuit" near a state park. The spate of drone sightings in the skies above New Jersey have caused alarm among state lawmakers, prompting one to call for a "limited state of emergency … until the public receives an explanation" regarding the source of the unidentified drones.

New Jersey State Senator Jon Bramnick joins'America Reports' to discuss recent mysterious drone sightings in New Jersey. A New Jersey state senator is calling on the Department of Defense to investigate the recent mysterious nighttime drone sightings amid rising public frustration over a lack of answers. "Let me be clear: The state police, this is way beyond their expertise … We know the Department of Defense has the technology to monitor these drones," State Sen. Jon Bramnick, R-N.J., told co-anchor John Roberts Wednesday on "America Reports." "The problem is we don't have the Department of Defense in New Jersey at this time. And that's what I call for. Until the Department of Defense comes in, shuts down airspace completely to drones, do a limited state of emergency – no drones in the sky until we figure out what's going on here," Bramnick warned.

'Fox & Friends Weekend' co-host Rachel Campos-Duffy opens up about seeing drones outside her home. U.S. lawmakers from New Jersey joined in many residents' frustrations over dozens of reports of drones being flown near sensitive sites like a military research facility in recent weeks, and they are now calling on federal agencies to immediately help investigate and address the escalating issue. Rep. Chris Smith, R-N.J., joined law enforcement leaders in his district on Monday on Long Beach Island, having been one of the key figures leading efforts to investigate the source and possible risks associated with the drone activity. "I have been speaking with Ocean County Sheriff Mike Mastronardy, Monmouth County Sheriff Shaun Golden, and national security officials located in the area to discuss the widespread reports of unidentified drone activity across my central New Jersey congressional district and across our state," Smith said in a statement. "Understandably, New Jersey residents are very alarmed at this significant and reoccurring phenomenon – and the tepid response from our state and federal agencies so far is totally unacceptable. As we saw with the Chinese spy balloon last year, our fiercest adversaries will stop at nothing to surveil our homeland and threaten our national security."

Artificial intelligence generated content (AIGC), a rapidly advancing technology, is transforming content creation across domains, such as text, images, audio, and video. Its growing potential has attracted more and more researchers and investors to explore and expand its possibilities. This review traces AIGC's evolution through four developmental milestones-ranging from early rule-based systems to modern transfer learning models-within a unified framework that highlights how each milestone contributes uniquely to content generation. In particular, the paper employs a common example across all milestones to illustrate the capabilities and limitations of methods within each phase, providing a consistent evaluation of AIGC methodologies and their development. Furthermore, this paper addresses critical challenges associated with AIGC and proposes actionable strategies to mitigate them. This study aims to guide researchers and practitioners in selecting and optimizing AIGC models to enhance the quality and efficiency of content creation across diverse domains.

The increasing use of transformer-based large language models brings forward the challenge of processing long sequences. In document visual question answering (DocVQA), leading methods focus on the single-page setting, while documents can span hundreds of pages. We present GRAM, a method that seamlessly extends pre-trained single-page models to the multi-page setting, without requiring computationally-heavy pretraining. To do so, we leverage a single-page encoder for local page-level understanding, and enhance it with document-level designated layers and learnable tokens, facilitating the flow of information across pages for global reasoning. To enforce our model to utilize the newly introduced document-level tokens, we propose a tailored bias adaptation method. For additional computational savings during decoding, we introduce an optional compression stage using our C-Former model, which reduces the encoded sequence length, thereby allowing a tradeoff between quality and latency. Extensive experiments showcase GRAM's state-of-the-art performance on the benchmarks for multi-page DocVQA, demonstrating the effectiveness of our approach.